fix(policies): start a policy's handlers before its new inputs - #801

Summary

A policy on a pcap_file tap saw none of its packets in pktvisord. In the orb-agent lab for MON-444, a 140-packet capture applied through the admin API counted 0 in both the net and dns handlers, and nothing was exported.

Cause. PolicyManager::load created and started each new input stream's <stream>-resources policy before it started the user's policy. Starting the resources policy started the input. Policy::start() starts its handlers before its inputs, but by then the input had already run. A pcap_file, and a dnstap_file, is read whole inside start(), so it was consumed before the user's handlers were connected. A live interface lost only its first milliseconds, which is why this went unnoticed.

Fix.

• Order: every handler on a new input now runs before the input starts.

  • create_resources_policy creates, registers and starts the resources handler, but not the resources policy.
  • Then policy_ptr->start() starts the user's handlers and then the inputs.
  • Then each resources policy is started and registered. Its handler and its input are already running, and every start() returns early when already running.

  Both the user's handlers and the resources handler see the whole file.

• Rollback: Policy::start() now stops the handlers it started when a later handler or input fails.

  • The new order starts the handlers before a new input can fail.
  • A sequence handler can't be removed while the next one is still connected to it.
  • Without the rollback, a failed load threw "unable to remove, handler has consumers" from the cleanup. The policy's handlers stayed registered, and every retry got 409 "already exists".

• Tidying: create_resources_policy now returns the unstarted resources policy, and it lost its unused policy_name parameter. Errors are wrapped once by the caller, so messages are unchanged. The resources handler is registered before its policy, which closes a window where a failed handler registration left a registered policy pointing at a destroyed handler.

Tests

• Good Config pcap_file input feeds the policy's handlers: a pcap tap on tests/fixtures/dns_ipv4_udp.pcap and a net policy.
  • It checks 140 events and the capture's start timestamp, which failed on develop (0 == 140).
  • It also checks that the resources policy runs and that its handler recorded CPU and memory samples, which failed with the first commit of this PR (null != nullptr).
• Bad Config: exception on input start with sequence modules: a pcap tap with a missing pcap_file and sequence modules [dns, net]. Two loads must both fail with policy [broken_view] failed to start: Cannot open pcap/pcapng file, leaving no policy or handler behind. Without the Policy::start() rollback it failed with "unable to remove, handler has consumers".
• The existing mock input-start test also checks that the policy is gone and that a retry fails the same way.
• Suite: all 25 ctest targets pass on macOS (Release).
• On a rebuilt pktvisord, through the admin API:
  • pcap_file: the policy counts net 140 and dns 140 (70 queries, 70 replies), against 0 on develop. A DELETE and re-POST counts 140 again. The resources policy records cpu_usage and memory_bytes samples.
  • Broken sequence policy, single tap and multi-tap: a clean 500 twice. Nothing is left but the existing policy, and the shared input's consumer count returns to where it was.

Not addressed here (found in review, all older than this change)

• Dangling policy pointer on a reused input: a load that fails before the policy has started leaves the policy pointer on a reused, running input. Its policy count stays one too high, so that input and its resources policy are never torn down.
• Lock order: load takes the input map and then the policy map, while remove_policy takes them the other way round, so a concurrent POST and DELETE can deadlock. This change shortens the window: an input's start() no longer runs while the input map lock is held.
• Narrow catch in load: it catches std::runtime_error only. A std::logic_error, such as PcapPlusPlus's std::invalid_argument, skips the rollback. With this change the leftover handlers are at least stopped.
• Live capture by interface name: separately, it fails with PcapPlusPlus 25.05. PcapInputStream::start() assigns the iface string to an IPv4Address and an IPv6Address, and 25.05 throws on anything that isn't an address, so iface: eth0 gives "Not a valid IPv4 address: eth0". Seen in the orb-agent develop image on Linux and on macOS builds. It is unrelated to this change and needs its own fix.

🤖 Generated with Claude Code